Robot device with changing dialogue and control method therefor and storage medium

ABSTRACT

Sentences corresponding to internal statuses of a robot device or the like are created and uttered, thereby expressing the internal statuses. The robot device or the like comprise means for recognizing an external status, and means for generating an emotion based on the internals status, whereby a change in the emotion is reflected upon a dialogue. The internal status is not associated with a sentence, but it exists independently of the system and is always varied depending on various external inputs and internal changes of the system. Accordingly, even when the same question is made on the robot device or the like, the contents of a reply are changed depending on the internal status at that time, and a manner of providing a reply also differs depending on the internal status.

TECHNICAL FIELD

The present invention relates to a dialogue device and method and astorage medium product for use with a walking robot device or amulti-articulated robot, which operates in response to commands fromusers and/or ambient environments, or with animation characters, etc.operated based on computer graphics (CG) (i.e., with a robot device orthe like). More particularly, the present invention relates to a robotdevice, a control method for the robot device, and a storage mediumproduct, the robot device being capable of creating sentences, utteringthe sentences and communicating with users through conversations.

Stated otherwise more exactly, the present invention relates to a robotdevice capable of creating and uttering sentences depending on emotionsof the robot device or the like, a control method for the robot device,and a storage medium product. In particular, the present inventionrelates to a robot device capable of creating and uttering sentencesdepending on internal statuses of the robot device or the like, therebyexpressing the internal status, a control method for the robot device,and a storage medium product.

BACKGROUND ART

Hitherto, a variety of ideas have been proposed and developed on awalking robot device or a multi-articulated robot, which operates inresponse to commands from users and/or ambient environments, or onanimation characters, etc. operated based on computer graphics (CG).Those robot devices or animation characters (hereinafter referred toalso as a “robot device or the like” collectively) perform a series ofactions in response to commands issued from users.

For example, a robot device having the shape and structure designed inimitation of a four-footed walking animal such as a dog, i.e., theso-called “pet robot”, lies sprawled in response to a command (e.g.,voice input) “Lie down!” from a user, or raises its paw to “shake hands”when the user puts a hand in front of the robot mouth.

Such a robot device or the like designed in imitation of actual animals,including a dog and a human being, are desirably capable of behaving ina manner as close as possible to actions and emotional expressions ofgenuine animals. It is also desired that the robot device or the like isable to not only perform just predetermined actions in response tocommands from users and/or external environments, but also behaveautonomously like genuine animals. This is because users get tiredeasily if the robot device or the like repeatedly performs the sameactions far away from actual situations, and the ultimate purpose ofdevelopment of the robot device or the like, i.e., cohabitation withhuman beings under the same living environments, cannot be achieved.

Recent intelligent robot devices or the likes incorporate therein thefunctions of voice input/output, voice recognition, voice synthesis,etc., and are able to converse or dialogue with users on the voicebasis. In the case of performing the conversation or utterance, it issimilarly desired that the robot devices or the likes are able to notonly perform just predetermined actions in response to commands fromusers and/or external environments, but also behave autonomously likegenuine animals.

Prior conversation utterance systems are proposed in, e.g., JapaneseUnexamined Patent Application Publication Nos. 10-247194, 8-339446 and9-16800.

Japanese Unexamined Patent Application Publication No. 10-247194discloses an automatic interpreting device in which translation isperformed and voices are synthesized while holding consistency as thewhole of a sentence by searching for an appropriate example regarding adifference in the sentence, e.g., an erroneous utterance intent of atranslated part. Also, Japanese Unexamined Patent ApplicationPublication No. 8-339446 discloses a dialogue device in which a varietyof emotions of a user are detected and information corresponding to thedetected emotion is issued from the computer side so that the user canfeel friendlier toward the device. Further, Japanese Unexamined PatentApplication Publication No. 9-16800 discloses a voice dialogue systemwith a face image, which is easily adaptable for changes in speech themeand is capable of providing practical and natural dialogues with users.

However, those prior conversation utterance systems are basicallyintended to recognize voices of speakers or users, to extract emotionsfrom facial expressions, and to create sentences in match with theemotions of the speakers only along the topics presented from thespeakers.

Also, the voice dialogue system with a face image, disclosed in JapaneseUnexamined Patent Application Publication No. 9-16800, is a system inwhich contents of replies corresponding to utterances are defined in theform of a table beforehand. This is hence no more than that replies andcorresponding emotions are decided beforehand, although the contents ofreplies include emotional information.

An intelligent robot device or the like has its internal statusesincluding emotions, etc. and is able to realize communication with usersat a deeper level by outputting the internal statuses to the outside.

In conventional robot devices or the likes, however, means forexpressing internal statuses are restricted to only actions of fourlegs, etc., and expressions cannot be easily recognized by everyone at aglance.

DISCLOSURE OF INVENTION

It is an object of the present invention to provide a superior robotdevice, a control method for the robot device, and a storage mediumproduct, which are applicable to robot devices or the likes, such as awalking robot device or a multi-articulated robot, which operates inresponse to commands from users and/or ambient environments, oranimation characters, etc. operated based on computer graphics (CG).

Another object of the present invention is to provide a superior robotdevice capable of creating and uttering sentences depending on emotionsof the robot device or the like, a control method for the robot device,and a storage medium product.

Still another object of the present invention is to provide a superiorrobot device capable of creating and uttering sentences depending oninternal statuses of the robot device or the like, thereby expressingthe internal statuses, a control method for the robot device, and astorage medium product.

The present invention has been accomplished with the view of achievingthe above objects, and according to a first aspect thereof, there isprovided a robot device having the function of dialoguing with users ora control method for the robot device, comprising:

external status input means or step for inputting an external status;

internal status managing means or step for managing an internal status;and

dialogue control means or step for reflecting a change of the internalstatus upon a dialogue with a user.

With the robot device or the control method for the robot deviceaccording to the first aspect of the present invention, the robot deviceor the like can recognize an external status, change an internal status,such as an emotion, and reflect a change of the internal status upon adialogue. The internal status is not associated with a sentence, but itexists independently of the system and is always varied depending onvarious external inputs and internal changes of the system.

Therefore, even when the same question is made on the robot device orthe like, the contents of a reply are changed depending on the internalstatus at that time, and a manner of providing a reply also differsdepending on the internal status. Thus, the robot device or the like canexpress its internal statuses using words.

As a result, anyone is able to easily confirm the internal status of therobot device or the like. For example, anyone is able to understand aliking of the robot or to make deeper communication with the robot. Itis also possible to enlarge the extent of dialogues between human beingsand the robot device or the like, to increase familiarity therebetween,and to facilitate empathy so that users feel more familiar with therobot device or the like. Consequently, cohabitation between humanbeings and the robot device or the like can be realized with more ease.

Further, the robot device or the like can express, using words, emotionsvarying upon interaction with the real world. Accordingly, a robotdevice or the like being more easily understandable and more familiarwith everyone can be designed and manufactured. It is hence possible toenhance amusingness and entertainingness of the robot device or thelike.

Moreover, since sentences are uttered depending on internal statuses,the robot device or the like is able to create a sentence in response tonot only an external stimulus, i.e., a question from the user, but alsovarious internal stimuli or internal statuses.

In addition, the present invention can also be implemented as onevariation such that factors for changes of the internal status arereflected upon the sentences. Additionally, sentences reflecting theinternal statuses and sentences explaining the internal statuses canalso be created on the basis of information regarding verbs andadjectives as well as pronouns.

According to a second aspect of the present invention, there is provideda robot device having the function of dialoguing with users or a controlmethod for the robot device, comprising:

external status input means or step for inputting an external status;

external status recognizing means or step for recognizing the inputtedexternal status;

external status understanding means or step for understanding a meaningand implications of the recognized external status in the sentence form;

internal status managing means or step for managing an internal status;

dialogue sentence creating means or step for creating a dialoguesentence based on a recognized result of the external status incomparison with the internal status; and

output means or step for externally outputting the created dialoguesentence.

With the robot device or the control method for the robot deviceaccording to the second aspect of the present invention, the robotdevice or the like can recognize an external status, change an internalstatus, such as an emotion, and reflect a change of the internal statusupon a dialogue. The internal status is not associated with a sentence,but it exists independently of the system and is always varied dependingon various external inputs and internal changes of the system.

Therefore, even when the same question is made on the robot device orthe like, the contents of a reply are changed depending on the internalstatus at that time, and a manner of providing a reply also differsdepending on the internal status. Thus, the robot device or the like canexpress its internal statuses using words.

In this connection, the internal status is constituted, for example, asan emotion of the robot device. Then, the internal status managing meansor step changes the internal status in response to a change of theexternal status.

Also, the robot device or the like may further include a database inwhich results obtained by classifying external statuses are stored alongwith internal statuses. In that case, the dialogue sentence creatingmeans or step can create a dialogue sentence using the database.

Moreover, the robot device or the control method for the robot devicemay further comprise action/behavior creating means or step for creatingan action or a behavior of a robot body based on the recognized resultof the external status in comparison with the internal status; and robotbody control means or step for realizing the created action or behaviorof the robot body. In that case, the robot device or the like is able toexpress a change of the internal status by using not only a dialogue butalso an action of the robot body.

According to a third aspect of the present invention, there is provideda dialogue device for dialoguing with users or a dialog control method,comprising:

external status input means or step for inputting an external status;

internal status managing means or step for managing an internal status;and

dialogue control means or step for reflecting a change of the internalstatus upon a dialogue with a user.

With the dialogue device or the dialog control method according to thethird aspect of the present invention, the dialogue device can recognizean external status, change an internal status, such as an emotion, andreflect a change of the internal status upon a dialogue. The internalstatus is not associated with a sentence, but it exists independently ofthe system and is always varied depending on various external inputs andinternal changes of the system.

Therefore, even when the same question is made on the dialogue device,the contents of a reply are changed depending on the internal status atthat time, and a manner of providing a reply also differs depending onthe internal status. Thus, the dialogue device can express its internalstatuses under use.

According to a fourth aspect of the present invention, there is provideda dialogue device for dialoguing with users or a dialog control method,comprising:

external status input means or step for inputting an external status;

external status recognizing means or step for recognizing the inputtedexternal status;

external status understanding means or step for understanding a meaningand implications of the recognized external status in the sentence form;

internal status managing means or step for managing an internal status;

dialogue sentence creating means or step for creating a dialoguesentence based on a recognized result of the external status incomparison with the internal status; and

output means or step for externally outputting the created dialoguesentence.

With the dialogue device or the dialog control method according to thefourth aspect of the present invention, the dialogue device canrecognize an external status, change an internal status, such as anemotion, and reflect a change of the internal status upon a dialogue.The internal status is not associated with a sentence, but it existsindependently of the system and is always varied depending on variousexternal inputs and internal changes of the system.

Therefore, even when the same question is made on the dialogue device,the contents of a reply are changed depending on the internal status atthat time, and a manner of providing a reply also differs depending onthe internal status. Thus, the dialogue device can express its internalstatuses under use.

In this connection, the internal status managing means or step maychange the internal status in response to a change of the externalstatus.

Also, the dialogue device may include a database in which resultsobtained by classifying external statuses are stored along with internalstatuses. In that case, the dialogue sentence creating means or step cancreate a dialogue sentence using information stored in the database.

According to a fifth aspect of the present invention, there is provideda storage medium product physically storing, in the computer readableform, computer software described so as to execute, on a computersystem, control of a robot device having the function of dialoguing withusers, the computer software comprising the steps of:

an external status input step of inputting an external status;

an external status recognizing step of recognizing the inputted externalstatus;

an external status understanding step of understanding a meaning andimplications of the recognized external status in the sentence form;

an internal status managing step of managing an internal status;

a dialogue sentence creating step of creating a dialogue sentence basedon a recognized result of the external status in comparison with theinternal status; and

an output step of externally outputting the created dialogue sentence.

According to a sixth aspect of the present invention, there is provideda storage medium product physically storing, in the computer readableform, computer software described so as to execute a dialogue processwith users on a computer system, the computer software comprising thesteps of:

an external status input step of inputting an external status;

an external status recognizing step of recognizing the inputted externalstatus;

an external status understanding step of understanding a meaning andimplications of the recognized external status in the sentence form;

an internal status managing step of managing an internal status;

a dialogue sentence creating step of creating a dialogue sentence basedon a recognized result of the external status in comparison with theinternal status; and

an output step of externally outputting the created dialogue sentence.

The storage medium products according to the fifth and sixth aspects ofthe present invention are each a medium product for providing, in thecomputer readable form, computer software to, for example, a universalcomputer system capable of executing various program codes. That mediumproduct is in the form of a storage medium, such as a CD (Compact Disc),an FD (Flexible Disc) or an MO (Magneto-Optical Disc), which can becarried with users and detachably attached to the system. Alternatively,it is technically possible to provide the computer software to aparticular computer system via a transmission medium, such as a network(regardless of whether the network is wireless or wired).

The storage medium product defines the structural or functionalcooperative relationship between the computer software and the storagemedium for realizing the predetermined functions of the computersoftware on the computer system. In other words, by installing thepredetermined computer software in the computer system with the aid ofthe storage medium products according to the fifth and sixth aspects ofthe present invention, the cooperative actions are developed on thecomputer system so as to provide similar operations and advantages tothose obtainable with the robot device and the control method for therobot device according to the second aspect of the present invention andwith the robot device and the control method for the robot deviceaccording to the fourth aspect of the present invention.

Still other objects, features and advantages of the present inventionwill be apparent upon reading a detailed description given below inconnection with an embodiment of the present invention and theaccompanying drawings.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a view showing an external appearance of a locomotion robot 1walking with four legs, in which the present invention is embodied.

FIG. 2 is a block diagram schematically showing a hardware configurationof a control unit for controlling the operation of the locomotion robot1.

FIG. 3 is a block diagram schematically showing a functionalconfiguration of a controller 32.

FIG. 4 is a block diagram schematically showing a functionalconfiguration of a signal recognizing unit 41.

FIG. 5 is a block diagram schematically showing a functionalconfiguration of an emotion and instinct model module 42.

FIG. 6 is a chart schematically showing a dialogue mechanism 70 in thelocomotion robot 1 according to the embodiment.

FIG. 7 is a chart showing one example of the operation of the dialoguemechanism 70 executed when the locomotion robot 1 is asked from a userand replies responsively.

FIG. 8 is a chart showing one example of the operation of the dialoguemechanism 70 executed when the locomotion robot 1 recognizes an objectand then speaks.

BEST MODE FOR CARRYING OUT THE INVENTION

An embodiment of the present invention will be described below in detailwith reference to the drawings.

FIG. 1 shows an external appearance of a locomotion robot 1 walking withfour legs, in which the present invention is embodied. A shown, thelocomotion robot 1 is a multi-articulated locomotion robot modeled onthe shape and structure of a four-legged animal. In particular, thelocomotion robot 1 of this embodiment is a pet robot designed inimitation of the shape and structure of a dog as a typical example ofpet animals, and is able to not only cohabit with human beings underhuman living environments, for example, but also express actions inresponse to user operations.

The locomotion robot 1 comprises a trunk unit 2, a head unit 3, a tail4, and four legs, i.e., leg units 6A-6D.

The trunk unit 2 accommodates a control unit (not shown in FIG. 1 anddescribed later) for controlling the operation of a robot body in asupervising manner, and a battery 21 (not shown in FIG. 1) serving as amain power source for the robot body.

The head unit 3 is disposed substantially at an upper front end of thetrunk unit 2 through a neck joint 7 having degrees of freedom indirections of roll, pitch and yaw axes (as shown). Also, the head unit 3incorporates therein an image recognizing unit 15 corresponding to“eyes” of a dog, such as a CCD (Charge Coupled Device) camera, amicrophone 16 corresponding to “ears”, a speaker 17 corresponding to a“mouth”, a touch sensor 18 corresponding to a tactile sense, a pluralityof LED indicators 19, and a remote controller receiving unit 20 forreceiving commands transmitted from a user through a remote controller(not shown). In addition, the head unit 3 may further include sensorsconstituting five senses of a living body.

The tail 4 is attached to substantially an upper rear end of the trunkunit 2 in a bendable or rotatable manner through a tail joint 8 havingdegrees of freedom in directions of roll and pitch axes.

The leg units 6A and 6B constitute forelegs, and the leg units 6C and 6Dconstitute hind legs. The leg units 6A-6D are constituted ascombinations of thigh units 9A-9D and shank units 10A-10D, respectively,and are attached to four corners of the underside of the trunk unit 2 atfront and rear ends on both left and right sides. The thigh units 9A-9Dare coupled to the corresponding predetermined positions of the trunkunit 2 through crotch joints 11A-11D each having degrees of freedom indirections of roll, pitch and yaw axes. Also, the thigh units 9A-9D andthe shank units 10A-10D are coupled to each other through knee joints12A-12D each having degrees of freedom in directions of roll and pitchaxes.

In the legged locomotion robot 1 constructed as shown, by drivingrespective joint actuators in accordance with commands from thelater-described control unit, the robot is able to perform actions suchas swinging the head unit 3 vertically or horizontally, waving the tail4, and walking or running through synchronous cooperative driving of thefoot units 6A-6D.

Degrees of freedom in movement of each joint of the locomotion robot 1are in fact realized by rotating a joint actuator (not shown) providedfor each axis. The number of degrees of freedom in movement of eachjoint of the locomotion robot 1 is a matter of choice in design andshould not be construed as limiting the gist of the present invention.

FIG. 2 schematically shows a hardware configuration of a control unitfor controlling the operation of the locomotion robot 1.

The head 3 includes a command receiving unit 30 made up of themicrophone 16 and the remote controller receiving unit 20, externalsensors 31 including the image recognizing unit 15 and the touch sensor18, the speaker 17, and the LED indicators 19.

The trunk unit 2 accommodates the battery 21 serving as a main powersource, the controller 32 for controlling the overall operation of thelocomotion robot 1 in a supervising manner, and internal sensors 35including a battery sensor 33 for detecting the remaining capacity ofthe battery 21 and a heat sensor 34 for detecting the heat generatedinside the robot body.

Further, a plurality of actuators 23A-23N are disposed in respectiveportions (e.g., driven joint portions) of the locomotion robot 1.

The command receiving unit 30 is made up of the remote controllerreceiving unit 20 and the microphone 16, and is able to commands, e.g.,“Walk!”, “Lie down!”, and “After ball!”, which are given to thelocomotion robot 1 from the user.

The remote controller receiving unit 20 receives commands inputted froma remote controller (not shown) with user operations, produces areception signal S1A, and sends it to the controller 32. Those commandsfrom the remote controller are transferred using, for example, thetechnique of near-range wireless data communication such as infrareddata communication (IrDA), Bluetooth or IEEE802.11b.

When the user utters voices corresponding to any desired command, themicrophone 16 collects the voices, produces a voice signal S1B, andsends it to the controller 32.

The touch sensor 18 as one of the external sensors 31 detects actions,such as “stroking” and “hitting”, imposed on the locomotion robot 1 fromthe user. When the user imposes a desired action, for example, bytouching the touch sensor 18, the touch sensor 18 produces acorresponding touch detection signal S2A, and sends it to the controller32.

The image recognizing unit 15 as another one of the external sensors 31detects ambient environmental information, such as “dark” and “there isa favorite toy”, as a result of recognizing environments around thelocomotion robot 1, or detects movements of other locomotion robots,such as “another robot is running”. The image recognizing unit 15 sends,to the controller 32, an image signal S2B obtained as a result ofpicking up an image of the surroundings.

The, internal sensors 35 detect internal statuses of the locomotionrobot 1 itself, such as “feel hungry”, which means lowering of theremaining capacity of the battery, and “have a fever”. The batterysensor 33 and the heat sensor 34 are included in the internal sensors35.

The battery sensor 33 detects the remaining capacity of the battery 21serving as a main power source of the robot body, and sends a resultingbattery capacity detection signal S3A to the controller 32. Also, theheat sensor 34 detects the heat generated inside the robot body, andsends a resulting heat detection signal S3B to the controller 32.

In accordance with command signals S1 supplied from the commandreceiving unit 30, external information signals S2 supplied from theexternal sensors 31, and internal information signals S3 supplied fromthe internal sensors 35, the controller 32 produces control signalsS5A-S5N for driving the actuators 23A-23N and sends them to theactuators 23A-23N, respectively, thereby controlling the operation ofthe robot body in a supervising manner.

On that occasion, the controller 32 produces, as required, a voicesignal S10 and a luminous signal S11 which are outputted to the outside.The voice signal S10 is outputted to the outside through the speaker 17,and the luminous signal S11 is sent to the LED indicators 19 for issuinga desired luminous output (such as blinking or change in color) so thatinformation representing, e.g., the internal status of the robot bodycan be fed back to the user. The user can be notified of, for example,the emotion of the locomotion robot 1 with the luminous output. Insteadof the LED indicators 19, an image display (not shown) for displaying animage may be provided. This enables the locomotion robot to presentinformation regarding its own emotion, etc. to the user more exactly andclosely through a desired image displayed on the display.

Control of the robot body by the controller 32 will be described below.

FIG. 3 schematically shows a functional configuration of the controller32. As shown in FIG. 3, the controller 32 comprises various modules,i.e., a signal recognizing module 41, an emotion and instinct modelmodule 42, an action deciding mechanism module 43, a posture transitionmechanism module 44, and a control mechanism module 45. These modulescan be in practice realized with software processing carried out on acomputer through execution of predetermined program code.

The signal recognizing module 41 executes processing to recognize andunderstand various input signals representing the internal statuses andthe external environments. FIG. 4 schematically shows a functionalconfiguration of the signal recognizing unit 41. A following descriptionis made of the functions of the signal recognizing module 41 withreference to FIG. 4.

The touch sensor 18 comprises, e.g., contact sensors and pressuresensitive sensors which are dispersedly arranged over the whole of thelocomotion robot 1. For example, when the user applies an externalforce, which implies “stroking” or “hitting”, to the locomotion robot 1,a resulting sensor input signal S2A is supplied to a meaning converter52. The meaning conversion processor 51 interprets or infers the meaningand implications involved in the user operation, such as “stroking” or“hitting”, and outputs an obtained result to a language processor 56 asmeaning converted data in the text form.

The image recognizing unit 15 is constituted as an image pickup device,e.g., a CCD (Charge Coupled Device). An image input signal S2B producedupon capturing, for example, actions and behaviors (including gestures)of the user is supplied to a meaning conversion processor 52. Themeaning conversion processor 56 interprets or infers the meaning andimplications involved in the actions and behaviors of the user, andoutputs an obtained result to the language processor 56 as meaningconverted data in the text form. Also, when entering, as an image,visual identifying information (such as “cyber codes” and landmarks)that has the meaning defined in the language form beforehand, themeaning conversion processor 52 is able to execute meaning conversionprocessing uniquely corresponding to the result of image recognition.

A voice input signal S1B from the microphone 16 is produced from thehead. A voice recognition processor 53 recognizes, as a text, voices ofthe user inputted through the voice input unit 53 and outputs it to thelanguage processor 56.

The battery sensor 33 detects the remaining capacity of the battery 21serving as a main power source of the robot body, and a resultingbattery capacity detection signal S3A is supplied to a meaningconversion processor 54. The meaning conversion processor 54 interpretsor infers the meaning and implications involved in the remainingcapacity of the battery 21, such as “feel hungry”, and outputs anobtained result to the language processor 56 as meaning converted datain the text form.

The heat sensor 34 detects the heat generated inside the robot body, anda resulting heat detection signal S3B is supplied to a meaningconversion processor 55. The meaning conversion processor 55 interpretsor infers the meaning and implications involved in the detected heat ofthe robot body, such as “have a fever”, and outputs an obtained resultto the language processor 56 as meaning converted data in the text form.

The language processor 56 analyzes the meaning and implications involvedin the inputted results from the external sensors 31, the internalsensors 35, etc. while utilizing a language database 57, and understandsthe inputted results in the language or sentence form. Also, thelanguage processor 56 sends recognized results of input information fromthe external sensors 31, the internal sensors 35, etc. to the emotionand instinct model module 42 in the sentence or another form.

The language database 57 accumulates, in the form of a database, thematters that can be recognized or expressed by the locomotion robot 1.In this embodiment, as described later, the language database 57 managesa database containing results obtained through classification ofexternal statuses along with emotions. The contents stored in thelanguage database 57 are updated with experiences and learning.

The language processor 56 processes pleasure/unpleasure and relatedimplications in the locomotion robot 1 regarding the recognizable orexpressible matters while referring to the language database 57. Also,the language processor 56 creates a sentence having the contents of amatched sentence through comparison of the recognizable or expressiblematters and words decomposed based on the implications. The createdsentence is expressed through an output device, e.g., the speaker 17.

Returning now to FIG. 3 again, the control by the controller 32 will bedescribed below. The emotion and instinct model module 42 handles in theform of models the emotions and instincts constituting the internalstatuses of the locomotion robot 1, and modifies emotion models andinstinct models depending on changes of environments inside and outsidethe robot body, which are detected through the signal recognizing unit41.

The action deciding mechanism module 43 decides a next action(behavior), which is to be executed by the locomotion robot 1, inaccordance with not only the command signal S1, the external informationsignal S2, and the internal information signal S3, but also an emotionand instinct status information S10 obtained from the emotion andinstinct model module 42.

The posture transition mechanism module 44 plans a posture transitionschedule for transition to the next action (behavior) decided by theaction deciding mechanism module 43. Additionally, information regardingthe action (behavior) decided by the action deciding mechanism module 43is fed back to the emotion and instinct model module 42 so that theemotion and instinct model module 42 can decide the status of emotion orinstinct with reference to the action (behavior) having been decided.

In accordance with posture transition information S18 sent from theposture transition mechanism module 44 based on the posture transitionschedule, the control mechanism module 45 controls the operations ofrespective driving units, such as the actuators 23A-23N, and thenactually transits the posture for executing the next action (behavior)decided by the action deciding mechanism module 43.

FIG. 5 schematically shows a functional configuration of the emotion andinstinct model module 42. As shown in FIG. 5, the emotion and instinctmodel module 42 is mainly divided into an emotion group 60 constitutingemotion models, and a desire group 61 constituting instinct modelsprepared as models having different attributes from those of the emotionmodels.

Herein, the emotion models are each a model constituted with an emotionparameter having a certain value to express one of the emotionsspecified for the locomotion robot 1 through an action depending on thevalue of the emotion parameter. The value of the emotion parameter isvaried up and down primarily in accordance with an external input signal(external factor) corresponding to a situation, such as being “hit” or“scolded”, which is detected by a sensor, e.g., a pressure sensor or avisual sensor. Of course, the emotion parameter is also changed inaccordance with an internal input signal (internal factor) representingthe remaining capacity of the battery or the temperature inside therobot body.

The instinct models are each a model constituted with an instinctparameter having a certain value to express one of the instincts(desires) specified for the locomotion robot 1 through an actiondepending on the value of the instinct parameter. The value of theinstinct parameter is varied up and down primarily in accordance with aninternal input signal representing a desire for, e.g., “doing exercises”based on action history or “recharging (being hungry)” based on theremaining capacity of the battery. Of course, the instinct parameter isalso changed in accordance with an external input signal (externalfactor) similarly to the emotion parameter.

Those groups of emotion models and instinct models are each made up ofplural kinds of models having the same attributes. More specifically,the emotion group 60 includes emotion units 60A-60F as independentemotion models having the same attributes, and the desire group 61includes desire units 61A-61D as independent desire models having thesame attributes.

The emotion group 60 includes, for example, an emotion unit 60Arepresenting “joy”, an emotion unit 60B representing “sorrow”, anemotion unit 60C representing “anger”, an emotion unit 60D representing“surprise”, an emotion unit 60E representing “fear”, and an emotion unit60F representing “dislike”. Also, the desire group 61 includes, forexample, a desire unit 61A representing a “movement instinct”, a desireunit 61B representing a “love instinct”, a desire unit 61C representinga “recharge instinct”, and a desire unit 61D representing a “searchinstinct”.

Each of the emotion units 60A-60F indicates the degree of emotion usingthe intensity of, for example, 0 to 100 levels (i.e., values of theemotion parameter), and changes the intensity of emotion moment bymoment in accordance with the command signal S1, the externalinformation signal S2, and the internal information signal S3 suppliedto the corresponding one. Thus, the emotion and instinct model module 42combines the intensities of the emotion units 60A-60F varying moment bymoment with each other, thereby expressing the emotional state of thelocomotion robot 1 and modeling emotion changes with time.

Further, desired ones of the emotion units influence with each other soas to vary their intensities. For example, the emotion units are coupledin a mutually suppressing manner or a mutually stimulating manner sothat the intensities of the emotion units are varied upon a mutualinfluence between them.

The locomotion robot 1 according to this embodiment can recognize theexternal status on the sentence basis, and can produce the emotion basedon the internal status, thus enabling emotion changes to be reflectedupon the dialogue with the user. The internal status referred to in thisembodiment is not associated with a sentence, but it existsindependently of the system and is always varied depending on variousexternal inputs and internal changes of the system. In other words,since the locomotion robot 1 performs an expressing operation such asuttering a sentence depending on its own internal status, the locomotionrobot is able to create a sentence in response to not only an externalstimulus, i.e., a question from the user, but also various internalstimuli or internal statuses, and to orally express the created sentencein the form of dialogue with the user. Accordingly, even when the samequestion is made on a robot device or the like, the contents of a replyare changed depending on the internal status at that time, and a mannerof providing a reply also differs depending on the internal status.

FIG. 6 schematically shows a dialogue mechanism 70 provided on thelocomotion robot 1 according to this embodiment. As shown in FIG. 6, thedialogue mechanism 70 is made up of a recognition object 71, anunderstanding object 72, the language database 57, an internal statusobject 73, a creation object 74, and an execution object 75.

The recognition object 71 takes inputs from hardware, such as themicrophone 16 and the touch sensor 18, into the system in therecognizable form.

The understanding object 72 gives meaning to the form taken in by therecognition object 71. For example, the understanding object 72 givesthe meaning of “HIT” to a pressure 80 or the meaning of “triangle” tothe presence of three angles.

The understanding object 72 refers to the language database 57 andsearches for a corresponding one of the recognizable or expressiblematters from the language database 57. The language database 57accumulates, in the form of a database, the matters that can berecognized or expressed by the locomotion robot 1. Preferably, thelanguage database 57 manages a database containing results obtainedthrough classification of external statuses along with emotions.Pleasure/unpleasure and related implications in the locomotion robot 1regarding each of the recognizable or expressible matters are alsoprocessed while referring to the language database 57. The languagedatabase 57 is updated with experiences and learning. Also, the languagedatabase 57 includes sentence templates and is able to create a sentencehaving the contents of a matched sentence through comparison of therecognizable or expressible matters and words decomposed based on theimplications.

The recognition object 71, the understanding object 72, and the languagedatabase 57 are constituted in the signal recognizing unit 41 (see FIG.3).

The internal status object 73 is constituted, for example, by theemotion and instinct model module 42 (see FIG. 3), and changes theinternal status of the locomotion robot 1 in accordance with theimplication provided by the understanding object.

The creation object 74 is constituted by the action deciding mechanismmodule 43 (see FIG. 3), and decides a next action (behavior) of thelocomotion robot 1 in accordance with outputs of the understandingobject 72 and the internal status object 73. In this embodiment, thecreation object 74 creates the next action in the form of a behaviorexpressed with body actions and a sentence.

The execution object 75 executes an object created by the creationobject 74 and outputs the created object to the outside. When thecreated object is an action, it is executed with operations of the robotbody through driving of the actuators 23A-23N. When the created objectis a sentence, it is executed with utterance through the speaker 17. Theexecution object is constituted by the posture transition mechanismmodule 44 and the control mechanism module 45 (see FIG. 3).

Further, the result of execution of the external output by the executionobject 75 is fed back to the internal status object 73 for changing theinternal status.

Examples of the operation of the dialogue mechanism 70 in the locomotionrobot 1 will be described below.

FIG. 7 is a chart showing one example of the operation of the dialoguemechanism 70 executed when the locomotion robot 1 is asked from a userand replies responsively.

The locomotion robot 1 includes a database storing recognizable andexpressible matters. When the user asks to the locomotion robot 1, “Doyou like a ball?”, the locomotion robot 1 collects the uttered humanwords with the microphone 16, performs voice recognition in therecognition object 71 based on the HMM (Hidden Markov Model), etc., anddecomposes the sentence depending on the recognized result (the HiddenMarkov Model is obtained by statistically modeling units of voices, suchas phonemes and words, and has a status transition structure). Further,the understanding object 72 determines that the voice input sentence isan interrogative one, what are the subject (S) and the predicate (V),respectively, and that the sentence asks for a liking to S (apparent S“ball” in Japanese).

In this example, the understanding object 72 takes “ball” out of thedatabase 57 and determines that the “ball” has a high score of pleasure,i.e., it is a “favorite”. The creation object 74 creates a reply “S+V”for the question “Do+S+V?”, i.e., “I like a ball”. Then, the executionobject 75 creates a train of pronunciations corresponding to the createdsentence, produces waveforms of the train of pronunciations, anddelivers the waveforms through the speaker 17. As a result, a dialogueof question and reply regarding a liking is realized.

Similarly, when “Why do you like a ball?” is inputted, the understandingobject 72 determines based on the sentence divided by the recognitionobject 71 that the inputted sentence is an interrogative one, what arethe subject (S) and the predicate (V), respectively, and that thesentence asks the reason why S+V.

In this example, the understanding object 72 takes “ball” out of thedatabase 57 and extracts the factor “able to kick”, which determines thescore of pleasure/unpleasure. Then, based on a sentence “S+V because OO”indicating the reason, the creation object 74 creates a reply sentence“S+V because I can kick it”. Thereafter, the execution object 75 createsa train of pronunciations corresponding to the created sentence,produces waveforms of the train of pronunciations, and delivers thewaveforms through the speaker 17. As a result, a dialogue of questionand reply regarding a liking is realized.

Not only a noun, but also a nominalized verb (such as “playing” and anominalized adjective (such as “the beautiful” or “the joyful”) can beused as the subject S (apparent one in Japanese). This expands the widthof sentence expressions.

Further, when the user asks, “Do you like it?”, while pointing at aball, the recognition object 71 divides the inputted sentence, andthereafter the understanding object 72 understands that “it”corresponding to S (apparent S in Japanese) in the inputted sentence isa demonstrative pronoun. Then, the locomotion robot 1 determines basedon an image input what the user points at, and combines “it” with“ball”. After combining a demonstrative pronoun with a particular thingin that way, processing is executed in the same manner as that describedabove.

FIG. 8 shows one example of the operation of the dialogue mechanism 70executed when the locomotion robot 1 recognizes an object with a camera.

When a recognizable thing, e.g., a ball, is put within the visual fieldof the locomotion robot 1, the locomotion robot 1 takes in an image ofthe thing with the camera 15. Then, the recognition object 71 determinesthat the taken-in image represents a ball.

The understanding object 72 takes out “ball” by referring to thedatabase 57 and determines that the “ball” has a high score of pleasure,i.e., it is a “favorite”.

Correspondingly, the creation object 74 creates a sentence of “S+V”,i.e., “I like a ball”. Then, the execution object 75 creates a train ofpronunciations corresponding to the created sentence, produces waveformsof the train of pronunciations, and delivers the waveforms through thespeaker 17. As a result, expression about a liking of the locomotionrobot 1 itself can be realized.

Finally, examples of sentences created with application of the dialoguemechanism 70 according to this embodiment are listed below.

-   (1) The locomotion robot asks, “What is this?”, upon looking at a    thing whose name is unknown.-   (2) It says, “This is OO.”, upon looking at a thing whose name is    known.-   (3) Based on the parameter values of the instinct model and the    emotion model in itself, it says, “OO is XX”.    -   Ex.) With a high score of “hunger (recharge)”, it says, “I am        hungry”.    -   Ex.) With a high score of “joy”, it says, “I am joyful”.-   (4) For a matter that satisfies the instinct or emotion of the    locomotion robot, it says, “I want to XX (OO)”.    -   Ex.) To satisfy the exercise instinct, it says, “I want to kick        a ball”.    -   Ex.) To relieve thirst, it says, “I want to drink water”.-   (5) In response to a word “OO!” uttered from a user, it understands    that the word represents an imperative sentence, and replies, “I    OO”.    Supplements

The present invention has been described above in detail in connectionwith a particular embodiment. It is, however, apparent that thoseskilled in the art can make modifications and substitutions on theabove-described embodiment without departing from the scope of thepresent invention.

While the embodiment in which the present invention is applied to a petrobot has been primarily explained in this specification, the gist ofthe present invention is not always limited to products called “robots”.In other words, the present invention is similarly applicable toproducts belonging to other industrial fields, e.g., toys, so long asthe product is a mechanical device that performs movements in imitationof human behaviors by utilizing electric or magnetic actions. As amatter of course, the present invention can be further applied toanimation characters, etc. operated based on computer graphics (CG).

Thus, the present invention has been disclosed above only by way ofexample, and should not be interpreted in a limiting way. The gist ofthe present invention should be judged in consideration of the scope ofthe claims.

INDUSTRIAL APPLICABILITY

According to the present invention, there are provided a superior robotdevice, a control method for the robot device, and a storage mediumproduct, which are applicable to robot devices or the likes, such as awalking robot device or a multi-articulated robot, which operates inresponse to commands from users and/or ambient environments, oranimation characters, etc. operated based on computer graphics (CG).

Also, according to the present invention, there are provided a superiorrobot device capable of creating and uttering sentences depending oninternal statuses of the robot device or the like, thereby expressingthe internal statuses, a control method for the robot device, and astorage medium product.

With the robot device and the control method for the robot deviceaccording to the present invention, the robot device or the like canexpress its internal statuses using words. As a result, anyone is ableto easily confirm the internal statuses of the robot device or the like.For example, anyone is able to understand a liking of the robot or tomake deeper communication with the robot. It is also possible to enlargethe extent of dialogues between human beings and the robot device or thelike, to increase familiarity therebetween, and to facilitate empathy sothat users feel more familiar with the robot device or the like.Consequently, cohabitation between human beings and the robot device orthe like can be realized with more ease.

Further, with the robot device and the control method for the robotdevice according to the present invention, the robot device or the likecan express, using words, emotions varying upon interaction with thereal world. Therefore, a robot device or the like being more easilyunderstandable and more familiar with everyone can be designed andmanufactured. It is hence possible to enhance amusingness andentertainingness of the robot device or the like.

With the robot device and the control method for the robot deviceaccording to the present invention, the internal status (emotions areregarded as being a part of the internal status) is not associated witha sentence, but it exists independently of the system and is alwaysvaried depending on various external inputs and internal changes of thesystem. Accordingly, even when the same question is made on a robotdevice or the like, the contents of a reply are changed depending on theinternal status of the robot device or the like at that time, and amanner of providing a reply also differs depending on the internalstatus. It is to be particularly put in mind that internal statuses arenot associated with sentences, but the sentences are created dependingon the internal statuses.

Moreover, with the robot device and the control method for the robotdevice according to the present invention, since sentences are uttereddepending on internal statuses, it is able to create a sentence inresponse to not only an external stimulus, i.e., a question from theuser, but also various internal stimuli or internal statuses. Thepresent invention can also be implemented as one variation such thatfactors for changes of the internal status are reflected upon thesentences. Additionally, sentences reflecting the internal statuses andsentences explaining the internal statuses can also be created on thebasis of information regarding verbs and adjectives as well as pronouns.

1. A robot device having the function of dialoguing with users,comprising: external status input means for inputting an externalstatus; internal status managing means for managing an internal statusbased on classification of external status along with emotions updatedwith experiences and learning; dialogue control means for reflecting achange of the internal status upon a dialogue with a user; and dialoguemeans for having a dialogue with the user.
 2. A robot device having thefunction of dialoguing with users, comprising: external status inputmeans for inputting an external status; external status recognizingmeans for recognizing the inputted external status; external statusunderstanding means for understanding a meaning and implications of therecognized external status in the sentence form; internal statusmanaging means for managing an internal status based on classificationof external status along with emotions updated with experiences andlearning; dialogue sentence creating means for creating a dialoguesentence based on a recognized result of the external status incomparison with the internal status; and output means for externallyoutputting the created dialogue sentence.
 3. A robot device according toclaim 2, wherein the internal status is constituted as an emotion ofsaid robot device.
 4. A robot device according to claim 2, wherein saidinternal status managing means changes the internal status in responseto a change of the external status.
 5. A robot device according to claim2, further comprising storage means for storing results obtained byclassifying external statuses along with internal statuses, saiddialogue sentence creating means creating a dialogue sentence usinginformation stored in said storage means.
 6. A robot device according toclaim 2, further comprising: action/behavior creating means for creatingan action or a behavior of a robot body based on the recognized resultof the external status in comparison with the internal status; and robotbody control means for realizing the created action or behavior of saidrobot body.
 7. A control method for a robot device having the functionof dialoguing with users, comprising the steps of: an external statusinput step of inputting an external status; an internal status managingstep of managing an internal status based on classification of externalstatus along with emotions updated with experiences and learning; adialogue control step of reflecting a change of the internal status upona dialogue with a user; and an output step of outputting the dialogue tothe user.
 8. A control method for a robot device having the function ofdialoguing with users, comprising the steps of: an external status inputstep of inputting an external status; an external status recognizingstep of recognizing the inputted external status; an external statusunderstanding step of understanding a meaning and implications of therecognized external status in the sentence form; an internal statusmanaging step of managing an internal status based on classification ofexternal status along with emotions updated with experiences andlearning; a dialogue sentence creating step of creating a dialoguesentence based on a recognized result of the external status incomparison with the internal status; and an output step of externallyoutputting the created dialogue sentence.
 9. A control method for arobot device according to claim 8, wherein the internal status isconstituted as an emotion of said robot device.
 10. A control method fora robot device according to claim 8, wherein said internal statusmanaging step changes the internal status in response to a change of theexternal status.
 11. A control method for a robot device according toclaim 8, wherein said dialogue sentence creating step creates a dialoguesentence using a database in which results obtained by classifyingexternal statuses are stored along with internal statuses.
 12. A controlmethod for a robot device according to claim 8, further comprising thesteps of: an action/behavior creating step of creating an action or abehavior of a robot body based on the recognized result of the externalstatus in comparison with the internal status; and a robot body controlstep of realizing the created action or behavior of said robot body. 13.A dialogue device for dialoguing with users, comprising: external statusinput means for inputting an external status; internal status managingmeans for managing an internal status based on classification ofexternal status along with emotions updated with experiences andlearning; dialogue control means for reflecting a change of the internalstatus upon a dialogue with a user; and dialogue means for having adialogue with the user.
 14. A dialogue device for dialoguing with users,comprising: external status input means for inputting an externalstatus; external status recognizing means for recognizing the inputtedexternal status; external status understanding means for understanding ameaning and implications of the recognized external status in thesentence form; internal status managing means for managing an internalstatus based on classification of external status along with emotionsupdated with experiences and learning; dialogue sentence creating meansfor creating a dialogue sentence based on a recognized result of theexternal status in comparison with the internal status; and output meansfor externally outputting the created dialogue sentence.
 15. A dialoguedevice according to claim 14, wherein said internal status managingmeans changes the internal status in response to a change of theexternal status.
 16. A dialogue device according to claim 14, furthercomprising storage means for storing results obtained by classifyingexternal statuses along with internal statuses, said dialogue sentencecreating means creating a dialogue sentence using information stored insaid storage means.
 17. A dialogue control method in dialoguing withusers, comprising the steps of: an external status input step ofinputting an external status; an internal status managing step ofmanaging an internal status based on classification of external statusalong with emotions updated with experiences and learning; a dialoguecontrol step of reflecting a change of the internal status upon adialogue with a user; and an output step of outputting the dialogue tothe user.
 18. A dialogue control method in dialoguing with users,comprising the steps of: an external status input step of inputting anexternal status; an external status recognizing step of recognizing theinputted external status; an external status understanding step ofunderstanding a meaning and implications of the recognized externalstatus in the sentence form; an internal status managing step ofmanaging an internal status based on classification of external statusalong with emotions updated with experiences and learning; a dialoguesentence creating step of creating a dialogue sentence based on arecognized result of the external status in comparison with the internalstatus; and an output step of externally outputting the created dialoguesentence.
 19. A dialogue method according to claim 18, wherein saidinternal status managing step changes the internal status in response toa change of the external status.
 20. A dialogue method for a robotdevice according to claim 18, wherein said dialogue sentence creatingstep creates a dialogue sentence using a database in which resultsobtained by classifying external statuses are stored along with internalstatuses.
 21. A storage medium product physically storing, in thecomputer readable form, computer software described so as to execute, ona computer system, control of a robot device having the function ofdialoguing with users, said computer software comprising the steps of:an external status input step of inputting an external status; anexternal status recognizing step of recognizing the inputted externalstatus; an external status understanding step of understanding a meaningand implications of the recognized external status in the sentence form;an internal status managing step of managing an internal status based onclassification of external status along with emotions updated withexperiences and learning; a dialogue sentence creating step of creatinga dialogue sentence based on a recognized result of the external statusin comparison with the internal status; and an output step of externallyoutputting the created dialogue sentence.
 22. A storage medium productphysically storing, in the computer readable form, computer softwaredescribed so as to execute a dialogue process with users on a computersystem, said computer software comprising the steps of: an externalstatus input step of inputting an external status; an external statusrecognizing step of recognizing the inputted external status; anexternal status understanding step of understanding a meaning andimplications of the recognized external status in the sentence form; aninternal status managing step of managing an internal status based onclassification of external status along with emotions updated withexperiences and learning; a dialogue sentence creating step of creatinga dialogue sentence based on a recognized result of the external statusin comparison with the internal status; and an output step of externallyoutputting the created dialogue sentence.